The present invention relates to a multiple pulse series generating device and method which can be applied to a primary random pulse series generating unit, and particularly relating to a multiple pulse series generating device and method which can be applied to a primary random pulse series generating unit which can be used in a random pulse series generating apparatus that can be used in a signal processing circuit network such as, for example, a neural network computer imitating a neural network.
There is a case where a so-called approximate calculating is performed in which approximate data is used, which approximate data is approximate to data that is actually produced, but which approximate data is not the exact data that is actually produced. In this case, generating of a random number is necessary to produce the above approximate data. A random pulse series generating device in a first related art which is used for generating a pseudo-random pulse series is, as shown in FIG. 1, comprised of: a pseudo-random-number generator 1; a register 2 for storing a plurality of pulse series density values; and addition/subtraction unit (ALU) 3 for examining differences between these pulse series density values. In an application of such a random pulse series generating device, many pseudo-random pulse series are needed in one system. Therefore the circuit scale of the addition/subtraction unit (ALU) has to be correspondingly large, and thus the circuit construction becomes very large. Thus, it becomes difficult to constitute a system of the random pulse series generating device by using one LSI. One such system to which the device is applied, which system needs many pseudo-random pulse series, is a neural network system. The random pulse series generating apparatus in the first related art and a second related art will now be described which description will be limited to describing the application of the device to a neural network system.
The "pulse series" is obtained as a result of a pulse density modulation being performed on a signal or on a numerical value, both the signal and value having certain magnitudes respectively, and thus such a pulse series expresses a certain magnitude of a numeral. Further, the term "pulse density" means a value obtained as a result of counting how many pulses are "1" from among pulses present in a frame, the term "frame" meaning a part of the pulse series obtained as a result of dividing the pulse series into the partial pulse series present in unit times respectively. In an example of the pulse density, in a case where the frame is "100", the corresponding pulse density is determined as a result of counting how many pulses are "1" when a hundred pulses are input. Further, the term "random pulse series" means a pulse series in which the pulses of "1" are generated randomly. Further, the term "pseudo-random pulse series" means a pulse series approximate to the random pulse series.
The neural network system will now be described briefly. A basic unit of information processing in a living body is a neuron. Functions of neurons are imitated, and "neuron imitation elements" resulting from this imitation to form a network. This resulting network is the so-called neural network. This neural network is made for the purpose of enabling a parallel processing of information.
Basic operations executed in the neural network system will now be described with reference to FIG. 2. In an example of the operations, input signals x1 through x5 are multiplied by corresponding weights W1 through W5 respectively, the multiplication results are then added together, and a non-linear process is then performed on this addition result, as shown in the following equation (1): ##EQU1##
Further, a function for executing the non-linear process is called a "sigmoid function", which function is expressed by the following equation (2): EQU f(x)=1/(1+e.sup.-x) (2)
There is a method for executing the above multiplication and addition calculations shown in the equation (1), and in this method, multipliers and adders are used. Normally, in the above neural network system, more than a thousand neurons are used, and from several tens to up to several hundreds of input signals are supplied to each neuron. Thus, the number of the above multiplication and addition calculations that has to be performed becomes huge in the entire neural network system. Therefore, it becomes difficult to realize a neural network system made of corresponding hardware.
There is a method for reducing the scale of hardware corresponding to the neural network system. In this method, the calculation result is not obtained as a result of using respective values of the input signals xi and the weight factors wi themselves as they are. Instead, in the method the above input signals and weight factors are indicated by the corresponding pulse densities, the multiplication calculations of the input signals and weight factors are replaced by corresponding logical multiplication calculations respectively, and the addition calculation of these multiplication calculations are replaced by corresponding logical addition result respectively. This method can be realized by a circuit shown in FIG. 3.
That is, in an example of the above method, the input signal x1 and the weight factor W1 are supplied to a logical multiplication device 31, both the input signal x1 and the weight factor W1 comprising the pulse series respectively, and these pulse series corresponding to the respective input signal x1 and the weight factor W1. Similarly, the input signals x2 through x5 and the weight factors W2 through W5 are supplied to other logical multiplication devices 32 through 35 respectively. Outputs of these logical multiplication devices 31 through 35 are supplied to a logical addition unit 36, and output data provided from the logical addition device 36 is output of the neuron corresponding to the devices of FIG. 3.
In the above replacement of the arithmetic multiplication calculation of the equation (1) into the corresponding logical multiplication calculation, in the case where the input signals and the weight factors are expressed by the corresponding pulse densities, the more random the corresponding pulses generated, the nearer the result of the logical multiplication is to the result of the arithmetic multiplication. Thus, and such a random pulse series generating device as the above random pulse series generating device 4 of FIG.1 has to be used for a generating apparatus for obtaining these pseudo-random pulse series.
In such a system as the neural network system that needs a great number of pseudo-random pulse series, a great number of above random pulse series generating devices 4 are needed accordingly. However, the ALU 3 constituting a part of each of these generating apparatuses 4 has to have a large scale equivalent to the large scale of the corresponding circuit. Thus, the entire system for generating the great number of pseudo-random pulse series has a huge scale equivalent to the corresponding circuit construction, and constructing the entire system by using one LSI is difficult. This is a problem.
For the purpose of solving this problem, the present applicant has proposed a random pulse series generating apparatus 48, in the second related art, shown in FIG. 4. This random pulse series generating apparatus 48 will be now described with reference to FIG. 4.
An up/down counter 46 is capable of being pre-loaded or a shift input, the up/down counter being of a four-bit type. ("Capable of being pre-loaded or shift input" means that one of two methods, a pre-loading method or a shift input method, can be selected for inputting data to the up/down counter. In the pre-loading method, all data is input at once synchronously with a clock signal, and in the shift input method, data is input to the counter in series synchronously with a clock signal.) Input data D0 through D3, the data being used for presetting the up/down counter 46, are supplied to the up/down counter 46. These input data D0 through D3 are latched by the up/down counter 46, when a clock signal provided to the up/down counter 46 rises, in a case where a control signal LOAD provided to the up/down counter 46 is "1".
The above input data D0 through D3 respectively correspond to the above weight factors which are to be stored in the register 2 of FIG. 1, and the data D0 through D3 weigh the pseudo-random pulse series, as output data, output from an OR device 44 constituting a part of the random pulse series generating apparatus 48. That is, the data D0 through D3 are data for determining the pulse density of the pseudo-random pulse series. (Hereinafter, data stored in the up/down counter 46 will be referred to as random pulse density determination data.) In the apparatus 48, a part for storing the random pulse density determination data is not comprised, for example, of a RAM (read only memory), but is instead comprised of a counter such as the up/down counter 46. This creates an advantage, namely, re-writing of the stored data is facilitated.
Each of AND devices 40 through 43 has first and second input terminals, output terminals A0 through A3 of the up/down counter 46 being connected to the first input terminals of these AND devices 40 through 43 respectively. A primary random pulse series generating unit 47 generates the pseudo-random pulse series, outputs of this primary random pulse series generating unit 47 being connected to the second input terminals of the AND devices 40 through 43, respectively.
Further, output terminals of the above AND devices 40 through 43 are connected to four input terminals of an OR device 44, respectively, which OR device 44 provides the pseudo-random pulse series.
The above primary random pulse series generating unit 47 is comprised of four sets of random pulse series generating devices as shown in FIG. 5, each of the random pulse series generating devices of FIG. 5 having the same constitution as that of the random pulse series generating device 4 shown in FIG. 1. Respective registers 2-0 through 2-3 store the respective weight factors, the respective weight factors corresponding to respective primary random pulse series w0 through w3 which are provided from the respective ALUs 3. Thus, similarly to the above first related art, the pulse densities of the above primary random pulse series w0 through w3 may vary as a result of varying the stored data stored in the above registers 2-0 through 2-3 respectively.
Operations of the random pulse series generating apparatus 48 having the above composition will be now described.
This random pulse series generating apparatus 48 operates in a manner basically, similar to the operations described with reference to FIG. 4. That is, the up/down counter 46 provides, to the respective AND devices 40 through 43 in parallel, respective bit data of the random pulse density determination data, synchronous with the clock signal provided from the outside. The primary random pulse series w0 through w3 provided from the primary random pulse series generating unit 47 are provided to the AND devices 40 through 43 respectively, also synchronous with the above clock signal.
The AND devices 40 through 43 perform the logical-multiplication calculations on both the above data provided, synchronously with the above clock signal, respectively, and results of the logical-multiplication calculations are then provided to the OR device 44.
The OR device 44 performs the logical-addition calculations on the above logical-multiplied data provided from the AND devices 40 through 43 synchronous with the above clock signal, and the OR device 44 thus provides the output as one of the pseudo-random pulse series. Concrete examples of the respective output data will be described at appropriate later parts of this specification.
An advantage of the random pulse series generating apparatus of the second related art having the above mentioned composition of the apparatus 48 will now be described. That is, in an example of an application of random pulse series generating apparatuses, there may be a case where a hundred the pseudo-random pulse series are needed. In this case, in the above first related art, a hundred sets of the random pulse series generating devices, each generating device having the constitution shown in FIG. 1, are needed accordingly. That is, a hundred sets of the ALUs 3, each of which has a large circuit scale, are needed.
On the other hand, in a case where the random pulse series generating apparatus of the second related art, that has been proposed by the present applicant, is applied to the above example, the following constitution is needed for the random pulse series generating apparatus in the second related art for a hundred pseudo-random pulse series. A hundred sets, each set comprising the up/down counter 46 the AND devices 40 through 43 and the OR device 44, are provided, and the AND devices 40 through 43 of each set of the above hundred sets are provided with the primary random pulse series w0 through w3 provided from the primary random pulse series generating unit 47 commonly, in the random pulse series generating apparatus in the second related art for a hundred pseudo-random pulse series. Thus, the above advantage of the second related art is that it does not matter whether or not the primary random pulse generating unit 47 is provided in the random pulse series generating apparatus that has the above one hundred sets of parts 46, 40 through 43 and 44, as the random pulse series generating apparatus only has to have the four-bit constitution as in the above-mentioned example of FIG. 4. The above advantage means that the random pulse series generating apparatus in the second related art for a hundred pseudo-random pulse series has to have only the four sets of the above ALUs (each having the large circuit scale as mentioned above) that are to be provided in the primary random pulse series generating unit 47. A circuit scale of each of the hundred sets each comprising the up/down counter 46, AND devices 40 through 43, and the OR device 44 may be less than half the circuit scale of one set of the ALU. Thus, the random pulse series generating apparatus in the second related art for hundred pseudo-random pulse series has a circuit scale greatly reduced in size in comparison to the corresponding apparatus in the above first related art.
However, in the random pulse series generating apparatus 48 of the second related art, the primary random pulse series generating unit 47 has to have the plurality of the ALUs 3 as shown in FIG. 5. Thus, the circuit scale of the primary random pulse series generating unit 47 can become large depending on the number of the ALUs to be provided therein. This is a problem. For the purpose of solving this problem, preferably, the primary random pulse series generating unit 47 is provided separate from the random pulse series generating apparatus 48 or separate from the above random pulse series generating apparatus in the second related art for a hundred pseudo-random pulse series.